Image processing apparatus and operating method thereof

ABSTRACT

An image processing apparatus, including a memory configured to store one or more instructions; and at least one processor configured to execute the one or more instructions to: based on a first image and a probability model, optimize an estimated pixel value and estimated gradient values of each pixel of an original image corresponding to the first image, obtain an estimated original image based on the optimized estimated pixel value of the each pixel of the original image, obtain a decontour map based on the optimized estimated pixel value and the estimated gradient values of the each pixel of the original image, and generate a second image by combining the first image with the estimated original image based on the decontour map.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a bypass continuation of International ApplicationNo. PCT/KR2022/006494, filed on May 6, 2022, which is based on andclaims priority to Korean Patent Application No. 10-2021-0065089, filedon May 20, 2021, and Korean Patent Application No. 10-2022-0010225,filed on Jan. 24, 2022, in the Korean Intellectual Property Office, thedisclosures of which are incorporated by reference herein in theirentireties.

BACKGROUND 1. Field

Various embodiments of the disclosure relate to an image processingapparatus capable of improving image quality by using machine learning,and an operating method thereof.

2. Description of Related Art

As data traffic has increased in line with developments in computertechnologies, artificial intelligence has become an important trend forleading future innovation. Artificial intelligence emulates humanthinking, and thus, in effect, may be applied to all industrial fieldswithout limitations. Examples of artificial intelligence technologiesinclude pattern recognition, machine learning, an expert system, neuralnetworks, natural language processing, etc.

Machine learning is a field of artificial intelligence which allows amachine (e.g., a computer) to learn from data by itself, to recognizepatterns (e.g. regularities) in the data, and thus make a decision ordetermination even on information that is not present in the data, basedon the recognized patterns. Types of machine learning include supervisedlearning, unsupervised learning, reinforcement learning, or the like.

Machine learning may be variously used in the image processing field,and, in particular, may be used in processes for improving imagequality.

SUMMARY

Various embodiments of the disclosure may provide an image processingapparatus capable of removing a false contour included in an image byusing machine learning based on a probability model, and an operatingmethod of the image processing apparatus.

In accordance with an aspect of the disclosure, an image processingapparatus includes a memory configured to store one or moreinstructions; and at least one processor configured to execute the oneor more instructions to: based on a first image and a probability model,optimize an estimated pixel value and estimated gradient values of eachpixel of an original image corresponding to the first image, obtain anestimated original image based on the optimized estimated pixel value ofthe each pixel of the original image, obtain a decontour map based onthe optimized estimated pixel value and the estimated gradient values ofthe each pixel of the original image, and generate a second image bycombining the first image with the estimated original image based on thedecontour map.

The first image may include a true contour which is included in theoriginal image, and a false contour which is not included in theoriginal image, and the second image may be an image from which thefalse contour is removed.

The at least one processor may be further configured to execute the oneor more instructions to: obtain, based on the first image, initialvalues of the estimated pixel value and the estimated gradient values ofthe each pixel of the original image, and optimize the estimated pixelvalue and the estimated gradient values of the each pixel of theoriginal image by updating the estimated pixel value and the estimatedgradient values of the each pixel of the original image based on theprobability model.

The probability model may be obtained by modeling a probability that apixel value of each pixel included in a first region around a firstpixel of the first image originates from a first pixel of the originalimage having the estimated pixel value and the estimated gradientvalues, and the at least one processor may be further configured toexecute the one or more instructions to optimize the estimated pixelvalue and the estimated gradient values of the first pixel by updatingthe estimated pixel value and the estimated gradient values of the firstpixel such that the probability increases.

The at least one processor may be further configured to execute the oneor more instructions to: obtain pixel values of pixels included in asecond region around the first pixel in the original image, based on theestimated pixel value and the estimated gradient values of the firstpixel, and optimize the estimated pixel value and the estimated gradientvalues of the first pixel by updating the estimated pixel value and theestimated gradient values of the first pixel such that a loss functiondetermined based on the probability is minimized, and the probabilitymay be represented by a function of a difference between a pixel valueof each pixel included in the second region and a pixel value of theeach pixel included in the first region.

The at least one processor may be further configured to execute the oneor more instructions to: obtain a texture map based on the first imageand the estimated original image, obtain a curvature map based on theoptimized estimated gradient values of the each pixel of the originalimage, and obtain the decontour map based on the texture map and thecurvature map.

The at least one processor may be further configured to execute the oneor more instructions to generate the texture map by obtaining adifference image between the first image and the estimated originalimage and performing filtering on the difference image.

The at least one processor may be further configured to execute the oneor more instructions to obtain the curvature map by computing acurvature of each pixel of the estimated original image based on theoptimized estimated gradient values of the each pixel of the originalimage.

The texture map may represent a first weight of the each pixel of theoriginal image, wherein the curvature map represents a second weight ofthe each pixel of the original image, wherein the decontour map mayrepresent a third weight of the each pixel of the original image, andthe at least one processor may be further configured to execute the oneor more instructions to obtain the third weight based on the firstweight and the second weight of the each pixel of the original image.

The at least one processor may be further configured to execute the oneor more instructions to: receive a third image including a frame imagesubsequent to the first image, obtain first estimated informationincluding the estimated pixel value and the estimated gradient values ofthe each pixel of the original image, which are optimized and subsampledwith respect to the first image, obtain second estimated information bysubsampling initial values of an estimated pixel value and estimatedgradient values of each pixel of the third image, obtain differenceinformation between the first image and the third image, obtain thirdestimated information based on the first estimated information and thedifference information, obtain fourth estimated information by combiningthe second estimated information with the third estimated informationbased on a first probability that the third image originates from thesecond estimated information and a second probability that the thirdimage originates from the third estimated information, obtain fifthestimated information by performing optimization on the fourth estimatedinformation based on the subsampled third image, and generate a fourthimage from which a false contour of the third image is removed, based onan estimated pixel value and estimated gradient values included in thefifth estimated information.

The at least one processor may be further configured to execute the oneor more instructions to: obtain a third probability that a false contourof the first image is included in each pixel of the first image, andobtain the fourth estimated information by combining the secondestimated information and the third estimated information based on thefirst probability, the second probability, and the third probability.

The at least one processor may be further configured to execute the oneor more instructions to: obtain an estimated original image of the thirdimage by upscaling the estimated pixel value included in the fifthestimated information, obtain a texture map based on the third image andthe estimated original image of the third image, obtain a curvature mapbased on the estimated gradient values included in the fifth estimatedinformation, upscale the curvature map, generate a decontour map of thethird image based on the texture map and the upscaled curvature map, andgenerate the fourth image based on the decontour map of the third image.

In accordance with an aspect of the disclosure, an operating method ofan image processing apparatus includes, based on a first image and aprobability model, optimizing an estimated pixel value and estimatedgradient values of each pixel of an original image corresponding to thefirst image; obtaining an estimated original image based on theoptimized estimated pixel value of the each pixel of the original image;obtaining a decontour map based on the optimized estimated pixel valueand the estimated gradient values of the each pixel of the originalimage; and generating a second image by combining the first image withthe estimated original image based on the decontour map.

The first image may include a true contour which is included in theoriginal image, and a false contour which is not included in theoriginal image, and the second image may be an image from which thefalse contour is removed.

The optimizing, based on the first image and the probability model, ofthe estimated pixel value and the estimated gradient values of the eachpixel of the original image may include: obtaining, based on the firstimage, initial values of the estimated pixel value and the estimatedgradient values of the each pixel of the original image; and optimizingthe estimated pixel value and the estimated gradient values of the eachpixel of the original image by updating the estimated pixel value andthe estimated gradient values of the each pixel of the original imagebased on the probability model.

The probability model may be obtained by modeling a probability that apixel value of each pixel included in a first region around a firstpixel of the first image originates from a first pixel of the originalimage having the estimated pixel value and the estimated gradientvalues, and the optimizing of the estimated pixel value and theestimated gradient values of the each pixel of the original image mayinclude optimizing the estimated pixel value and the estimated gradientvalues of the first pixel by updating the estimated pixel value and theestimated gradient values of the first pixel such that the probabilityincreases.

The optimizing of the estimated pixel value and the estimated gradientvalues of the each pixel of the original image may include: obtainingpixel values of pixels included in a second region around the firstpixel in the original image, based on the estimated pixel value and theestimated gradient values of the first pixel; and optimizing theestimated pixel value and the estimated gradient values of the firstpixel by updating the estimated pixel value and the estimated gradientvalues of the first pixel such that a loss function determined based onthe probability is minimized, and the probability may be represented bya function of a difference between a pixel value of each pixel includedin the second region and a pixel value of the each pixel included in thefirst region.

The obtaining of the decontour map may include: obtaining a texture mapbased on the first image and the estimated original image; obtaining acurvature map based on the optimized estimated gradient values of theeach pixel of the original image; and obtaining the decontour map basedon the texture map and the curvature map.

The obtaining of the texture map may include: obtaining a differenceimage between the first image and the estimated original image; andgenerating the texture map by performing filtering on the differenceimage.

The obtaining of the curvature map may include obtaining the curvaturemap by computing a curvature of each pixel of the estimated originalimage based on the optimized estimated gradient values of the each pixelof the original image.

The texture map may represent a first weight of the each pixel of theoriginal image, the curvature map may represent a second weight of theeach pixel of the original image, the decontour map may represent athird weight of the each pixel of the original image, and the obtainingof the decontour map may include obtaining the third weight based on thefirst weight and the second weight of the each pixel of the originalimage.

The operating method may further include receiving a third imageincluding a frame image subsequent to the first image; obtaining firstestimated information including the estimated pixel value and theestimated gradient values of the each pixel of the original image, whichare optimized and subsampled with respect to the first image; obtainingsecond estimated information by subsampling initial values of anestimated pixel value and estimated gradient values of each pixel of thethird image; obtaining difference information between the first imageand the third image; obtaining third estimated information based on thefirst estimated information and the difference information; obtainingfourth estimated information by combining the second estimatedinformation with the third estimated information based on a firstprobability that the third image originates from the second estimatedinformation and a second probability that the third image originatesfrom the third estimated information; obtaining fifth estimatedinformation by performing optimization on the fourth estimatedinformation based on the subsampled third image; and generating a fourthimage from which a false contour of the third image is removed, based onan estimated pixel value and estimated gradient values included in thefifth estimated information.

The operating method further may include obtaining a third probabilitythat a false contour of the first image is included in each pixel of thefirst image, and the obtaining of the fourth estimated information mayinclude obtaining the fourth estimated information by combining thesecond estimated information and the third estimated information basedon the first probability, the second probability, and the thirdprobability.

The generating of the fourth image from which the false contour of thethird image is removed, based on the estimated pixel value and theestimated gradient values included in the fifth estimated information,may include: obtaining an estimated original image of the third image byupscaling the estimated pixel value included in the fifth estimatedinformation; obtaining a texture map based on the third image and theestimated original image of the third image; obtaining a curvature mapbased on the estimated gradient values included in the fifth estimatedinformation; upscaling the curvature map; generating a decontour map ofthe third image based on the texture map and the upscaled curvature map;and generating the fourth image based on the decontour map of the thirdimage.

In accordance with an aspect of the disclosure, a non-transitorycomputer-readable recording medium is configured to store instructionswhich, when executed by at least one processor, cause the at least oneprocessor to: based on a first image and a probability model, optimizean estimated pixel value and estimated gradient values of each pixel ofan original image corresponding to the first image; obtain an estimatedoriginal image based on the optimized estimated pixel value of the eachpixel of the original image; obtain a decontour map based on theoptimized estimated pixel value and the estimated gradient values of theeach pixel of the original image; and generate a second image bycombining the first image with the estimated original image based on thedecontour map.

In accordance with an aspect of the disclosure, an image processingapparatus includes a memory configured to store one or moreinstructions; and at least one processor configured to execute the oneor more instructions to: based on a first image and a probability model,optimize an estimated pixel value and estimated gradient values of apixel of an original image corresponding to the first image, obtain anestimated original image based on the optimized estimated pixel value ofthe pixel, obtain a decontour map based on the optimized estimated pixelvalue and the estimated gradient values of the pixel, and generate asecond image by combining the first image with the estimated originalimage based on the decontour map.

The first image may include a true contour which is included in theoriginal image, and a false contour which is not included in theoriginal image, and the second image may include the true contour, andmay not include the false contour.

An image processing apparatus according to an embodiment of thedisclosure may remove a false contour included in an input image byusing a probability model, thereby improving image quality.

A false contour removal process according to an embodiment of thedisclosure may be applied to a real-time moving image, and the amount ofcomputation may be reduced by reusing estimated values of a previousframe image to process a current frame image.

Also, the size of memory and the amount of computation may be reduced byusing estimated values of a subsampled previous frame image.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating an operation, performed by an imageprocessing apparatus, of processing an input image, according to anembodiment.

FIG. 2 is a diagram for describing a method, performed by an originalimage estimator, of obtaining estimated information regarding anoriginal image, according to an embodiment.

FIG. 3 is a diagram for describing a probability model for modeling aprobability that a pixel value of a first image originates from a pixelvalue of an original image, according to an embodiment.

FIG. 4 is a diagram illustrating an operation of optimizing estimatedinformation regarding an original image, according to an embodiment.

FIG. 5 is a diagram illustrating a method of obtaining initial values ofestimated information regarding an original image, according to anembodiment.

FIG. 6 is a diagram illustrating an operation of detecting a falsecontour, according to an embodiment.

FIG. 7 is a diagram illustrating a method of generating a curvature map,according to an embodiment.

FIG. 8 is a diagram for describing a false contour removal operationaccording to an embodiment.

FIG. 9 is a block diagram of a configuration of an image processingapparatus according to another embodiment.

FIG. 10 is a diagram for describing a method, performed by areconfigurer of FIG. 9, of reconfiguring initial values of estimatedinformation regarding an original image of a current frame image,according to an embodiment.

FIG. 11 is a flowchart of a method of operating an image processingapparatus according to an embodiment.

FIG. 12 is a block diagram of a configuration of an image processingapparatus according to an embodiment.

DETAILED DESCRIPTION

Throughout the disclosure, the expression “at least one of a, b or c”indicates only a, only b, only c, both a and b, both a and c, both b andc, all of a, b, and c, or variations thereof.

Examples of terms used herein will be briefly described, and then thedisclosure will be described in detail.

Although terms used in the disclosure are selected from among commonterms that are currently widely used in consideration of their functionsin the disclosure, the terms may be different according to an intentionof one of ordinary skill in the art, a precedent, or the advent of newtechnology. Also, in particular cases, the terms are discretionallyselected by the applicant of the disclosure, in which case, the meaningof those terms will be described in detail in the corresponding part ofthe detailed description. Therefore, the terms used herein are notmerely designations of the terms, but the terms are defined based on themeaning of the terms and content throughout the disclosure.

Throughout the disclosure, when a part “includes” a component, this maymean that the part may additionally include other components rather thanexcluding other components as long as there is no particular opposingrecitation. Also, the terms described in the specification, such as “ .. . er (or)”, “ . . . unit”, “ . . . module”, etc., denote a unit thatperforms at least one function or operation, which may be implemented ashardware or software or a combination thereof.

Hereinafter, embodiments of the disclosure will be described in detailwith reference to the accompanying drawings. The disclosure may,however, be embodied in many different forms and should not be construedas being limited to the embodiments of the disclosure set forth herein.In order to clearly describe the disclosure, portions that are notrelevant to the description of the disclosure may be omitted, andsimilar reference numerals are assigned to similar elements throughoutthe specification.

FIG. 1 is a diagram illustrating an operation, performed by an imageprocessing apparatus, of processing an input image, according to anembodiment of the disclosure.

Referring to FIG. 1, the image processing apparatus 100 according to anembodiment of the disclosure may receive a first image 10 and processthe first image 10 to generate a second image 20. In embodiments, thefirst image 10 may include a false contour. A false contour is not anactual contour of an object included in an image, but is a contour thatis not present in an original image but is generated in the image due toquantization. For example, when quantization is performed in a processof encoding an original image, a false contour may be included in animage obtained by decoding the encoded image. Such false contours arehighly visible and thus have a large influence on the image quality, andaccordingly, a process of removing the false contours may be desired. Inaddition, a false contour may appear similar to a texture in an image,and thus, it is necessary to process the false contour differently froma texture region.

The image processing apparatus 100 may include an original imageestimator 110, a false contour detector 120, and a false contour remover130, and the original image estimator 110 may obtain estimatedinformation regarding the original image of the first image 10, based onthe first image 10 and a probability model. For example, the estimatedinformation regarding the original image may include an estimated pixelvalue, an estimated x-direction gradient value, which may be for examplea first estimated gradient value, and an estimated y-direction gradientvalue, which may be for example a second estimated gradient value, ofeach pixel included in the original image. An example of a method,performed by the original image estimator 110, of obtaining estimatedinformation regarding an original image will be described in detail withreference to FIGS. 2 to 5.

Also, the false contour detector 120 may detect a false contour based onthe first image 10 and the estimated information regarding the originalimage obtained by the original image estimator 110. For example, thefalse contour detector 120 may obtain an estimated image of the originalimage, which may be referred to for example as an estimated originalimage, and detect a false contour based on a difference between thefirst image 10 and the estimated original image. In embodiments, thefalse contour detector 120 may detect the degree of curve of a regionincluded in the image based on the estimated information regarding theoriginal image, and detect a false contour based on the degree of curveof the region. Here, the detected false contour may be represented by adecontour map. An example of a method, performed by the false contourdetector 120, of detecting a false contour will be described in detailwith reference to FIGS. 6 and 7.

The false contour remover 130 may perform decontouring to remove thefalse contour included in the first image 10 by using the decontour mapobtained by the false contour detector 120. The false contour remover130 may generate the second image 20 by combining the estimated originalimage obtained by the original image estimator 110 with the first image10. Here, the generated second image 20 may include only true contours,and may have a higher image quality than that of the first image 10.

An example of a method, performed by the false contour remover 130, ofremoving a false contour will be described in detail with reference toFIG. 8.

FIG. 2 is a diagram for describing a method, performed by an originalimage estimator, of obtaining estimated information regarding anoriginal image, according to an embodiment of the disclosure.

According to an embodiment of the disclosure, the original imageestimator 110 may obtain estimated information regarding the originalimage of the first image 10 (i.e., an input image). According to anembodiment of the disclosure, estimated information 210 regarding anoriginal image may include an estimated pixel value m, an x-directiongradient value g_(x), and a y-direction gradient value g_(y) of eachpixel included in the original image.

The original image estimator 110 may optimize the estimated information210 regarding the original image of the first image 10, based on thefirst image 10 and a probability model. In embodiments, the probabilitymodel may be determined by Bayes' rule. Bayes' rule describes therelationship between a prior probability and a posterior probability,and may be represented by Equation 1.

P(A|B)∝P(B|A)P(A)  [Equation 1]

In Equation 1, P(A) is a prior probability, and P(A|B) is a posteriorprobability. Here, A may denote the original image, and B may denote thefirst image 10 (i.e., an input image).

In the case where the intrinsic probability P(A) of the original imageis fixed and the first image 10 is given, the probability P(A|B) of theoriginal image being the original image of the first image 10 isproportional to the probability P(B|A) of the first image 10 originatingfrom the original image. Accordingly, by estimating an original imagehaving the highest probability of originating the first image 10, theoriginal image having the highest probability with respect to the firstimage 10 may be inferred.

The original image estimator 110 may estimate an original image havingthe highest probability of originating the first image 10 by performingoptimization on the probability model based on Bayes' rule.

For example, when the first image 10, which may be denoted as I, isgiven, a probability p(m, g_(x), g_(y)|I) of the original image, havingthe estimated information 210 is proportional to a probability p(I|m,g_(x), g_(y)) of the first image 10 originating from the original imagehaving the estimated information 210, and the probability modelaccording to an embodiment of the disclosure may be a model obtained bymodeling the probability p(I|m, g_(x), g_(y)) of the first image 10originating from the original image having the estimated information210. The original image estimator 110 may obtain optimized values of theestimated information 210 by updating the estimated information 210 byusing a derivative ∇p of the probability p(I|m, g_(x), g_(y)) such thatthe probability p(I|m, g_(x), g_(y)) has the maximum value.

The original image estimator 110 according to an embodiment of thedisclosure may obtain optimized values of estimated informationregarding each pixel included in an original image, and an example of aprobability model obtained by modeling the probability p(I|m, g_(x),g_(y)) for each pixel will be described in detail with reference to FIG.3.

FIG. 3 is a diagram for describing a probability model for modeling aprobability that a pixel value of a first image originates from a pixelvalue of an original image, according to an embodiment of thedisclosure.

The first image 10 according to an embodiment of the disclosure may bean image obtained by performing quantization on the original image.Quantization is a process of mapping continuous values of a signalincluded in an original image to quantized values. Accordingly, asillustrated in FIG. 3, a range 310 of pixel values included in theoriginal image may be estimated from a pixel value n of the first image10.

In addition, when quantizing the original image, it may be assumed thatnot only a signal of the original image but also a value to which randomnoise is applied (i.e., signal of original image+random noise) isquantized to obtain the first image 10, and the random noise has aparticular probability distribution. For example, when the random noisehas a Gaussian distribution N(0, σ²) and a value obtained by dividingthe size of the quantization interval by 2 is γ, a probabilityP(I[x]=n|M(x)=m) of the pixel value n of the first image originatingfrom a pixel value m of the original image may be represented byEquation 2.

$\begin{matrix}{{P\left( {{I\lbrack x\rbrack} = {n{❘{{M(x)} = m}}}} \right)} = {{P\left( {{{m(n)} - \gamma} \leq {m + \omega} < {{m(n)} + \gamma}} \right)} = {{\frac{1}{2}\left\{ {{\Phi\left( \frac{{m(n)} - m + \gamma}{\sigma\sqrt{2}} \right)} - {\Phi\left( \frac{{m(n)} - m - \gamma}{\sigma\sqrt{2}} \right)}} \right\}} = {\frac{1}{2}\left\{ {{\Phi\left( \frac{{\epsilon\left( {n,m} \right)} + \gamma}{\sigma\sqrt{2}} \right)} - {\Phi\left( \frac{{\epsilon\left( {n,m} \right)} - \gamma}{\sigma\sqrt{2}} \right)}} \right\}}}}} & \left\lbrack {{Equation}2} \right\rbrack\end{matrix}$

In addition, the probability P(I[x]=n|M(x)=m) may represent the area ofa first region 320 of FIG. 3, and the area of the first region 320increases as the difference between the pixel value m of the originalimage and a median m(n) of the quantization interval decreases.Accordingly, as ε(n, m), which denotes the difference between the medianm(n) and m, decreases, the probability that the pixel value n of thefirst image originates from the pixel value m of the original imageincreases. Also, referring to Equation 2, the probabilityP(I[x]=n|M(x)=m) may be represented by a function of ε(n, m).

FIG. 4 is a diagram illustrating an operation of optimizing estimatedinformation regarding an original image, according to an embodiment ofthe disclosure.

The original image estimator 110 according to an embodiment of thedisclosure may perform optimization on an estimated pixel value, anestimated x-direction gradient value (i.e., a first estimated gradientvalue), and an estimated y-direction gradient value (i.e., a secondestimated gradient value) of each pixel included in an original image,to obtain an optimized estimated pixel value, an optimized firstestimated gradient value, and an optimized second estimated gradientvalue.

The original image estimator 110 may perform filtering on the firstimage 10 to obtain an initial values m₀ of the estimated pixel value, aninitial value g_(x0) of the first estimated gradient value, and aninitial value g_(y0) of the second estimated gradient value of eachpixel included in the original image.

An example of this process will be described in detail with reference toFIG. 5.

FIG. 5 is a diagram illustrating a method of obtaining initial values ofestimated information regarding an original image, according to anembodiment of the disclosure.

Referring to FIG. 5, the original image estimator 110 may obtain initialvalues 510 of the estimated information regarding the original image byperforming filtering on the first image 10.

For example, the original image estimator 110 may obtain an initialvalue m₀ of an estimated pixel value of each pixel included in theoriginal image by performing convolution by applying a first convolutionfilter 521 to the first image 10.

In addition, the original image estimator 110 may obtain an initialvalue g_(x0) of a first estimated gradient value of each pixel includedin the original image by performing convolution by applying a secondconvolution filter 522 to the first image 10.

Furthermore, the original image estimator 110 may obtain an initialvalue g_(y0) of a second estimated gradient value of each pixel includedin the original image by performing convolution by applying a thirdconvolution filter 523 to the first image 10.

However, the method of obtaining initial values of estimated informationregarding an original image illustrated in FIG. 5 is merely an example,and the original image estimator 110 may obtain initial values ofestimated information regarding an original image by using variousmethods.

Referring again to FIG. 4, the original image estimator 110 may performoptimization on the estimated pixel value, the first estimated gradientvalue, and the second estimated gradient value, based on the initialvalues m₀, g_(x0), and g_(y0) of the estimated pixel value, the firstestimated gradient value, and the second estimated gradient value, andthe probability model described with reference to FIG. 3.

The original image estimator 110 may obtain estimated pixel values ofrespective pixels included in a first region 410 having a preset sizearound a first pixel (x₀, y₀) included in the original image, based onan initial pixel value m₀, an initial x-direction gradient value g_(x0)(i.e., an initial value of the first estimated gradient), and an initialy-direction gradient value g_(y0) (i.e., an initial value of the secondestimated gradient) of the first pixel (x₀, y₀).

For example, the original image estimator 110 may compute an estimatedpixel value m′ of a second pixel (x, y) included in the first region 410around the first pixel (x₀, y₀), according to Equation 3.

m′=m0+g _(x0)(x−x0)+g _(y0)(y−y0)  [Equation 3]

In Equation 3, m₀ denotes the initial pixel value of the first pixel(x₀, y₀), g_(x0) denotes the initial x-direction gradient value of thefirst pixel (x₀, y₀), and g_(y0) denotes the initial y-directiongradient value of the first pixel (x₀, y₀).

The original image estimator 110 may obtain a difference between theestimated original image and the first image 10. For example, theoriginal image estimator 110 may calculate a difference 430 between anestimated pixel value of each pixel included in the first region 410 ofthe estimated original image and a pixel value of each pixel included ina second region 420 of the first image 10. Here, the second region 420is a partial region of the first image 10 and corresponds to the firstregion 410.

A probability P({I}_(Ω)|m₀, g_(x) ₀ , g_(y) ₀ ) of the pixel valuesincluded in the second region 420 given that the pixel value, the firstgradient value, and the second gradient value of the first pixel (x₀,y₀) in the original image are m₀, g_(x0), and g_(y0), respectively, maybe represented by Equation 4.

$\begin{matrix}{{P\left( {\left\{ I \right\}_{\Omega}{❘{m_{0},g_{x_{0}},g_{y_{0}}}}} \right)} = {{{\prod_{{({x,y})} \in \Omega}{P\left( {{I\left\lbrack {x,y} \right\rbrack}{❘{{M\left( {x,y} \right)} = {m_{0} + {g_{x_{0}}\left( {x - x_{0}} \right)} + {g_{y_{0}}\left( {y - y_{0}} \right)}}}}} \right)}} = {\prod_{{({x,y})} \in \Omega}{\frac{1}{2}\left\{ {{\Phi\left( \frac{{\epsilon\left\lbrack {x,y} \right\rbrack} + \gamma}{\sigma\sqrt{2}} \right)} - {\Phi\left( \frac{{\epsilon\left\lbrack {x,y} \right\rbrack} - \gamma}{\sigma\sqrt{2}} \right)}} \right\}}}}}} & \left\lbrack {{Equation}4} \right\rbrack\end{matrix}$

In addition, a loss function L(m₀, g_(x) ₀ , g_(y) ₀ ) may berepresented by Equation 5.

$\begin{matrix}{{L\left( {m_{0},g_{x_{0}},g_{y_{0}}} \right)} = {{- {{\log P}\left( {\left\{ I \right\}_{\Omega}{❘{m_{0},g_{x_{0}},g_{y_{0}}}}} \right)}} = {- {\sum_{{({x,y})} \in \Omega}{\log\frac{1}{2}\left\{ {{\Phi\left( \frac{{\epsilon\left\lbrack {x,y} \right\rbrack} + \gamma}{\sigma\sqrt{2}} \right)} - {\Phi\left( \frac{{\epsilon\left\lbrack {x,y} \right\rbrack} - \gamma}{\sigma\sqrt{2}} \right)}} \right\}}}}}} & \left\lbrack {{Equation}5} \right\rbrack\end{matrix}$

The original image estimator 110 according to an embodiment of thedisclosure may update the estimated pixel value, the first estimatedgradient value, and the second estimated gradient value of the firstpixel (x₀, y₀) such that the loss function of Equation 5 decreases, inorder to perform optimization on the estimated pixel value, the firstestimated gradient value, and the second estimated gradient value of thefirst pixel (x₀, y₀).

In addition, the original image estimator 110 may use gradient descentto find an estimated pixel value, a first estimated gradient value, anda second estimated gradient value that minimize the loss function. Forexample, the original image estimator 110 may determine the estimatedpixel value, the first estimated gradient value, and the secondestimated gradient value of each pixel that minimize the loss function,by subtracting, from their previous values, products of a presetlearning rate and derivatives ∇L of the loss function, respectively.

Although only optimization of the first pixel (x₀, y₀) is illustrated inand described with reference to FIG. 4, the original image estimator 110may also perform optimization on remaining pixels included in theoriginal image in the same manner as that used for the first pixel (x₀,y₀), to obtain optimized values of the estimated pixel value, the firstestimated gradient value, and the second estimated gradient value ofeach pixel.

FIG. 6 is a diagram illustrating an operation of detecting a falsecontour according to an embodiment of the disclosure.

Referring to FIG. 6, the false contour detector 120 may receiveoptimized estimated information 610 regarding an original image from theoriginal image estimator 110. For example, the optimized estimatedinformation 610 may include an estimated pixel value m, a firstestimated gradient value g_(x), and a second estimated gradient valueg_(x) of each pixel included in the original image.

The false contour detector 120 may generate a texture map 620 based onthe estimated pixel values of the respectively pixels included in theoriginal image and the pixel values included in the first image 10. Forexample, the false contour detector 120 may generate the texture map 620by obtaining differences between the estimated pixel values of thepixels included in the original image and the pixel values included inthe first image 10, respectively, and performing filtering on thedifferences. For example, the false contour detector 120 may calculatethe absolute value of the difference of each pixel and use a medianfilter to calculate the median of the absolute values of respectivepixels included in a window region having a preset size. In addition,the false contour detector 120 may transform result values obtained byapplying the median filter, into values in the range of [0, 1] by usinga transformation function, to generate the texture map 620. However, thedisclosure is not limited thereto.

Here, the texture map 620 may represent a first weight of each pixel,and the first weights for pixels included in a texture region may bedetermined to be high, whereas the first weights for pixels included ina region other than the texture region may be determined to be low.

Also, the false contour detector 120 may generate a curvature map 630based on the first gradient value g_(x) and the second gradient valueg_(y) of each pixel included in the original image. For example, thefalse contour detector 120 may generate the curvature map 630 bycomputing Gaussian curvatures.

An example of this process will be described in detail with reference toFIG. 7.

FIG. 7 is a diagram illustrating a method of generating a curvature map,according to an embodiment of the disclosure.

Referring to FIG. 7, the false contour detector 120 may compute aGaussian curvature of each pixel by using the first gradient value g_(x)and the second gradient value g_(x) of each pixel included in theoriginal image.

For example, the false contour detector 120 may perform convolution byapplying a first filter 731 to first gradient values 710, which includeeach first gradient value g_(x) of each pixel included in the originalimage, to obtain x-direction derivatives g_(xx) of the first gradientvalues 710 of the respective pixels. In addition, the false contourdetector 120 may perform convolution by applying a second filter 732 tothe first gradient values 710, which include each first gradient valueg_(x) of each pixel included in the original image, to obtainy-direction derivatives g_(xy) of the first gradient values 710 of therespective pixels.

In addition, the false contour detector 120 may perform convolution byapplying the first filter 731 to second gradient values 720, whichinclude each second gradient value g_(y) of each pixel included in theoriginal image, to obtain x-direction derivatives g_(yx) of the secondgradient values 720 of the respective pixels.

In addition, the false contour detector 120 may perform convolution byapplying the second filter 732 to the second gradient values 720, whichinclude each second gradient value g_(y) of each pixel included in theoriginal image, to obtain y-direction derivatives g_(yy) of the secondgradient values 720 of the respective pixels.

The false contour detector 120 may compute a Gaussian curvature κ ofeach pixel by using Equation 6 and the obtained derivatives g_(xx),g_(xy), g_(yx), and g_(yy).

κ=g _(xx) *g _(yy) −g _(xy) *g _(yx)  [Equation 6]

Here, as the Gaussian curvature κ increases, the corresponding pixelregion has more curves, and thus is likely to be a texture region.

Referring again to FIG. 6, the curvature map 630 may represent a secondweight of each pixel, and a region with high second weights (i.e., highcurvatures) may have a high degree of or amount of curve, whereas aregion with low second weights (i.e., low curvatures) may have a lowdegree of or amount of curve.

The false contour detector 120 may generate a decontour map 640 based onthe texture map 620 and the curvature map 630. The decontour map 640 mayrepresent a third weight of each pixel, and the third weight may bedetermined based on the first weight of each pixel represented by thetexture map 620 and the second weight of each pixel represented by thecurvature map 630. For example, as the first weight and the secondweight of each pixel increase, the third weight may decrease, and, asthe first weight and the second weight of each pixel decrease, the thirdweight may increase. However, the disclosure is not limited thereto.

That is, a region with low third weights may be a texture region or aregion including true contours, while a region with high third weightsmay be a region including false contours.

FIG. 8 is a diagram for describing a false contour removal operationaccording to an embodiment of the disclosure.

Referring to FIG. 8, the false contour remover 130 may receive thedecontour map 640 from the false contour detector 120, and receive anestimated original image 810 from the original image estimator 110.

The false contour remover 130 may generate the second image 20 bycombining the estimated original image 810 with the first image 10 basedon the decontour map 640. For example, the false contour remover 130 maygenerate the second image 20 by applying, to the estimated originalimage 810, a third weight w3 of each pixel represented by the decontourmap 640 and applying a weight of (1−w3) to each pixel of the first image10 and then combining the estimated original image 810 with the firstimage 10. Here, the second image 20 may be an image from which falsecontours of the first image 10 are removed.

Meanwhile, the image processing apparatus according to an embodiment ofthe disclosure may also perform a process of removing false contoursfrom a moving image including a plurality of frame images. For example,in the case of a moving image that does not require real-time imageprocessing, the image processing apparatus 100 may perform the imageprocessing illustrated in and described with reference to FIGS. 1 to 8,on each of the plurality of frame images included in the moving image,to remove the false contours included in the plurality of frame images.

FIG. 9 is a block diagram of a configuration of an image processingapparatus 900 according to another embodiment of the disclosure.

The image processing apparatus 900 according to another embodiment ofthe disclosure may perform a false contour removal process on a movingimage that requires real-time image processing. For example, the imageprocessing apparatus may remove a false contour included in each of aplurality of frame images included in a real-time moving image.

Referring to FIG. 9, the image processing apparatus 900 may include anoriginal image estimator 910, a false contour detector 920, and a falsecontour remover 930. In addition, the original image estimator 910 mayinclude a first optimizer 911, a reconfigurer 912, and a secondoptimizer 913.

The first optimizer 911 may receive first estimated information (m1,g1_(x), g1_(y)) obtained by subsampling estimated values of an originalimage, which are optimized with respect to a previous frame imageI(t−1), and a subsampled previous frame image I′(t−1). The firstoptimizer 911 may perform optimization on first estimated values m1,g1_(x), and g1_(y) included in the first estimated information based onthe first estimated information (m1, g1_(x), g1_(y)) and the subsampledprevious frame image I′(t−1). An example of a method of optimizing firstestimated values has been described in detail with reference to FIG. 4,and thus detailed descriptions thereof will be omitted.

The first optimizer 911 may obtain optimized second estimated values m2,g2_(x), and g2_(y).

The reconfigurer 912 may receive second estimated information includingthe second estimated values m2, g2_(x), and g2_(y) obtained by the firstoptimizer 911, and reconfigure initial values of estimated informationregarding an original image of a current frame image I(t) based on thesecond estimated information.

An example of a method, performed by the reconfigurer 912, ofreconfiguring the initial values of the estimated information regardingthe original image of the current frame image I(t) will be described indetail with reference to FIG. 10.

FIG. 10 is a diagram for describing a method, performed by thereconfigurer 912 of FIG. 9, of reconfiguring initial values of estimatedinformation regarding an original image of a current frame image.

Referring to FIG. 10, the reconfigurer 912 may obtain third estimatedinformation 1020, which is a result of applying the amount of changebetween the previous frame image I(t−1) and the current frame image I(t)to second estimated information 1010 including second estimated valuesobtained by the first optimizer 911.

For example, the reconfigurer 912 may obtain difference information Δm′between initial estimated pixel values m′(t−1) of the original image ofthe previous frame image I(t−1) and initial estimated pixel values m′(t)of the original image of the current frame image I(t). Here, the initialestimated pixel values m′(t−1) of the original image of the previousframe image I(t−1) may be previously calculated and stored whenprocessing the previous frame image I(t−1). Also, the initial estimatedpixel values m′(t−1) of the original image of the previous frame imageI(t−1) and the initial estimated pixel values m′(t) of the originalimage of the current frame image I(t) may be subsampled values. Thereconfigurer 912 may obtain the third estimated information 1020 byapplying the difference information Δm′ to the second estimatedinformation 1010.

In addition, the reconfigurer 912 may obtain, based on the current frameimage I(t), initial values 1030 including m′(t), g′_(x)(t), andg′_(y)(t) of estimated information regarding the original image of thecurrent frame image I(t). The reconfigurer 912 may perform filtering onthe current frame image I(t) to obtain the initial values 1030 includingm′(t), g′_(x)(t), and g′_(y)(t) of the estimated information regardingthe original image. An example of a method of obtaining initial valuesof estimated information regarding an original image has been describedin detail with reference to FIG. 5, and thus detailed descriptionsthereof will be omitted.

The reconfigurer 912 may determine a weight w based on a probabilityp(I(t)|m′2, g2_(x), g2_(y)) of the current frame image I(t) originatingfrom an original image having the third estimated information 1020, aprobability p(I(t)|m′(t), g′_(x)(t), g′_(y)(t)) of the current frameimage I(t) originating from the initial values 1030 including m′(t),g′_(x)(t), and g′_(y)(t) of the estimated information regarding theoriginal image of the current frame image I(t), and a probability p1 ofa false contour existing in the previous frame image.

Here, the probability p(I(t)|m′2, g2_(x), g2_(y)) and the probabilityp(I(t)|m′(t), g′_(x)(t), g′_(y)(t)) may be represented by Equations 7and 8, respectively.

p(I(t)|m′2,g2x,g2y)=exp(−L(m′2,g2x,g2y;I(t)))  [Equation 7]

p(I(t)|m′(t),g′x(t),g′y(t))=exp(−L(m′(t),g′x(t),g′y(t);I(t)))  [Equation8]

The reconfigurer 912 may compute the weight w by using Equation 9.

$\begin{matrix}{w = {{p\left( {{m(t)},{{gx}(t)},{{{gy}(t)}{❘{I(t)}}}} \right)} = \frac{{\exp\left( {- \frac{L\left( {{m^{\prime}2},{g2x},{{g2y};{I(t)}}} \right)}{T}} \right)}*{p1}}{\begin{matrix}{{{\exp\left( {- \frac{L\left( {m^{\prime},{g2x},{{g2y};{I(t)}}} \right)}{T}} \right)}*{p1}} +} \\{{\exp\left( {- \frac{L\left( {{m^{\prime}(t)},{g^{\prime}{x(t)}},{{g^{\prime}{y(t)}};{I(t)}}} \right)}{T}} \right)}*\left( {1 - {p1}} \right)}\end{matrix}}}} & \left\lbrack {{Equation}9} \right\rbrack\end{matrix}$

Here, the probability p1 of a false contour existing in the previousframe image may be determined based on the second estimated information1010. For example, the reconfigurer 912 may compute a Gaussian curvatureW_(cntr) of each pixel by using the estimated gradient values includedin the second estimated information 1010. An example of a method ofcomputing the Gaussian curvature W_(cntr) of each pixel is illustratedin and described with reference to FIG. 7, and thus detaileddescriptions thereof will be omitted.

The reconfigurer 912 may obtain the probability p1 by applying atransformation function to the Gaussian curvature W_(cntr) of each pixelto map the Gaussian curvature W_(cntr) of each pixel to a value between0 and 1. For example, when the Gaussian curvature W_(cntr) is 3 orgreater, p1 may be mapped to 0, when the Gaussian curvature W_(cntr) is1.5 or less, p1 may be mapped to 1, and when the Gaussian curvatureW_(cntr) is greater than 1.5 and less than 3, p1 may be mapped to avalue between 0 and 1, and p1 may be linearly mapped to decrease as theGaussian curvature W_(cntr) increases. However, a method of obtainingthe probability p1 is not limited thereto, and the probability p1 may beobtained by using other methods.

When the weight w is determined, the reconfigurer 912 may obtain fourthestimated information (m(t), g_(x)(t), g_(y)(t)) by using Equation 8.

(m(t),gx(t),gy(t))=w*(mV),g′x(t),g′y(t))+(1−w)*(m′2,g2x,g2y)  [Equation10]

In Equation 10, (m′2, g2_(x), g2_(y)) denotes the third estimatedinformation 1020, (m′(t), g_(x)′(t), g_(y)′(t)) denotes the initialvalues of the estimated information regarding the original image of thecurrent frame image, and w denotes a weight computed by using Equation9.

Referring again to FIG. 9, the second optimizer 913 may upscale thefourth estimated information (m(t), g_(x)(t), g_(y)(t)) obtained by thereconfigurer 912, and obtain fifth estimated values by performingoptimization on fourth estimated values m(t), g_(x)(t), and g_(y)(t)included in the upscaled fourth estimated information by using thesubsampled current frame image I′(t). An example of a method ofoptimizing fourth estimated values has been described in detail withreference to FIG. 4, and thus detailed descriptions thereof will beomitted.

The false contour detector 920 may detect a false contour included inthe current frame image based on fifth estimated information includingfifth estimated values m5, g5_(x), and g5_(y) obtained by the secondoptimizer 913.

For example, the false contour detector 920 may include a texture mapgenerator 921, which may upscale the estimated values m5 of therespective pixels included in the fifth estimated information, andgenerate a texture map based on the upscaled estimated values m5 of therespective pixels and the current frame image I(t). An example of amethod of generating a texture map has been described in detail withreference to FIG. 6, and thus a detailed description thereof will beomitted.

The false contour detector 920 may also include a curvature mapgenerator 922, which may generate a curvature map based on first andsecond gradient values g5_(x) and g5_(x) of each pixel included in thefifth estimated information. An example of a method of generating acurvature map has been described in detail with reference to FIGS. 6 and7, and thus detailed descriptions thereof will be omitted.

The false contour remover 930 may upscale the curvature map generated bythe curvature map generator 922, and generate a decontour map based onthe upscaled curvature map and the texture map generated by the texturemap generator 921. An example of a method of generating a decontour maphas been described in detail with reference to FIG. 8, and thus adetailed description thereof will be omitted.

The false contour remover 930 may generate a reconstructed image bycombining the current frame image I(t) with the upscaled estimatedvalues m5 of the respective pixels, based on the decontour map. Anexample of a method of generating a reconstructed image from which falsecontours are removed has been described in detail with reference to FIG.8, and thus detailed descriptions thereof will be omitted.

FIG. 11 is a flowchart of a method of operating an image processingapparatus according to an embodiment of the disclosure.

Referring to FIG. 11, the image processing apparatus 100 according to anembodiment of the disclosure may optimize an estimated pixel value andan estimated gradient value of each pixel included in an original imageof a first image at operation S1110.

The image processing apparatus 100 may obtain initial values of theestimated pixel value and the estimated gradient values of each pixelincluded in the original image of the first image. For example, theimage processing apparatus 100 may perform filtering on the first imageto obtain initial values m₀, g_(x0), and g_(y0) of the estimated pixelvalue, the first estimated gradient value, and the second estimatedgradient value of each pixel included in the original image. An exampleof this has been described in detail with reference to FIG. 5, and thusa detailed description thereof will be omitted.

The image processing apparatus 100 may perform optimization on theestimated pixel value, the first estimated gradient value, and thesecond estimated gradient value, based on the initial values m₀, g_(x0),and g_(y0) of the estimated pixel value, the first estimated gradientvalue, and the second estimated gradient value, and the Bayes'rule-based probability model described with reference to FIG. 3.

The Bayes' rule-based probability model according to an embodiment ofthe disclosure may be obtained by modeling a probability that a pixelvalue of each pixel included in a first region around a first pixel in afirst image originates from a first pixel of an original image having anestimated pixel value, a first estimated gradient value, and a secondestimated gradient value.

The image processing apparatus 100 may optimize the estimated pixelvalue, the first estimated gradient value, and the second estimatedgradient value of the first pixel by updating the estimated pixel value,the first estimated gradient value, and the second estimated gradientvalue of the first pixel such that the probability increases.

The image processing apparatus 100 may obtain pixel values of pixelsincluded in a second region around the first pixel in the originalimage, based on the estimated pixel value and the estimated gradientvalues of the first pixel.

A loss function may be determined based on a probability that a pixelvalue of each pixel included in the first region of the first imageoriginates from a pixel value of each pixel included in the secondregion of the original image. Here, the probability and the lossfunction may be represented by a function of a difference between apixel value of each pixel included in the second region of the originalimage and a pixel value of each pixel included in the first region ofthe first image.

The image processing apparatus 100 may optimize the estimated pixelvalue, the first estimated gradient value, and the second estimatedgradient value of the first pixel by updating the estimated pixel value,the first estimated gradient value, and the second estimated gradientvalue of the first pixel such that the loss function is minimized.

In addition, the image processing apparatus 100 may perform optimizationon the remaining pixels included in the original image in the samemanner as that used for the first pixel, to obtain optimized values ofthe estimated pixel value, the first estimated gradient value, and thesecond estimated gradient value of each pixel included in the originalimage.

The image processing apparatus 100 according to an embodiment of thedisclosure may obtain an estimated original image based on the optimizedestimated pixel values of the respective pixels at operation S1120.

For example, the estimated original image may refer to an image, thepixel values of the pixels of which are the estimated pixel values,respectively.

The image processing apparatus 100 according to an embodiment of thedisclosure may obtain a decontour map based on the optimized estimatedpixel value and gradient values of each pixel at operation S1130.

The image processing apparatus 100 may generate a texture map based onthe estimated pixel values of the respective pixels included in theoriginal image and the pixel values included in the first image. Forexample, the image processing apparatus 100 may generate the texture mapby obtaining differences between the estimated pixel values of thepixels included in the original image and the pixel values included inthe first image, respectively, and performing filtering on thedifferences. Here, the texture map may represent a first weight of eachpixel, and the first weights for pixels included in a texture region maybe determined to be high, whereas the first weights for pixels includedin a region other than the texture region may be determined to be low.

Also, the image processing apparatus 100 may generate a curvature mapbased on the first gradient value and the second gradient value of eachpixel included in the original image. For example, the image processingapparatus 100 may generate the curvature map by computing Gaussiancurvatures. An example of a method of generating a curvature map hasbeen described in detail with reference to FIG. 7, and thus detaileddescriptions thereof will be omitted.

Here, the curvature map may represent a second weight of each pixel, anda region with high second weights (i.e., high curvatures) may have ahigh degree of or amount of curve, whereas a region with low secondweights (i.e., low curvatures) may have a low degree of or amount ofcurve.

The image processing apparatus 100 may generate a decontour map based onthe texture map and the curvature map. The decontour map may represent athird weight of each pixel, and the third weight may be determined basedon the first weight of each pixel represented by the texture map and thesecond weight of each pixel represented by the curvature map. Forexample, as the first weight and the second weight of each pixelincrease, the third weight may decrease, and, as the first weight andthe second weight of each pixel decrease, the third weight may increase.However, the disclosure is not limited thereto.

In embodiments, a region with low third weights may be a texture regionor a region including true contours, while a region with high thirdweights may be a region including false contours.

The image processing apparatus 100 according to an embodiment of thedisclosure may generate a second image by combining the first image withthe estimated original image based on the decontour map at operationS1140.

The image processing apparatus 100 may generate the second image byapplying, to the estimated original image, the third weight of eachpixel represented by the decontour map, applying (1− third weight) tothe first image, and then combining the first image with the estimatedoriginal image. Here, the second image may be an image from which falsecontours of the first image are removed.

FIG. 12 is a block diagram of a configuration of an image processingapparatus 1200 according to an embodiment of the disclosure.

Referring to FIG. 12, the image processing apparatus 1200 may include aprocessor 1210, a memory 1220, and a display 1230.

The image processing apparatus 1200 illustrated FIG. 12 may correspondto the image processing apparatus 100 illustrated in and described withreference to FIGS. 1 to 8 or the image processing apparatus 900illustrated in and described with reference to FIGS. 9 and 10.

The processor 1210 according to an embodiment of the disclosure maycontrol the overall operation of the image processing apparatus 1200.The processor 1210 according to an embodiment of the disclosure mayexecute one or more programs stored in the memory 1220.

The memory 1220 according to an embodiment of the disclosure may storevarious types of data, programs, or applications for driving andcontrolling the image processing apparatus 1200. The program stored inthe memory 1220 may include one or more instructions. The program (i.e.,one or more instructions) or application stored in the memory 1220 maybe executed by the processor 1210.

The processor 1210 according to an embodiment of the disclosure mayinclude at least one of a central processing unit (CPU), a graphicsprocessing unit (GPU), or a video processing unit (VPU). Alternatively,according to an embodiment of the disclosure, the processor 1210 may beimplemented as a system-on-a-chip (SoC) into which at least one of aCPU, a GPU, or a VPU. In embodiments, the processor 1210 may furtherinclude a neural processing unit (NPU).

The processor 1210 according to an embodiment of the disclosure mayoptimize estimated information regarding an original image of a firstimage based on a probability model to obtain optimized estimatedinformation, detect a false contour included in the first image based onthe optimized estimated information and the first image, and obtain asecond image from which the detected false contour is removed.

For example, the processor 1210 may perform at least one of theoperations of the original image estimator 110, the false contourdetector 120, or the false contour remover 130 illustrated in anddescribed with reference to FIGS. 1 to 8. In addition, the processor1210 may perform at least one of the operations of the first optimizer911, the reconfigurer 912, the second optimizer 913, the texture mapgenerator 921, the curvature map generator 922, and the false contourremover 930 illustrated in and described with reference to FIGS. 9 and10.

The processor 1210 may obtain initial values of the estimated pixelvalue and the estimated gradient values of each pixel included in theoriginal image of the first image. For example, the processor 1210 mayperform filtering on the first image to obtain initial values m₀,g_(x0), and g_(y0) of the estimated pixel value, the first estimatedgradient value, and the second estimated gradient value of each pixelincluded in the original image. An example of this has been described indetail with reference to FIG. 5, and thus detailed descriptions thereofwill be omitted.

The processor 1210 may perform optimization on the estimated pixelvalue, the first estimated gradient value, and the second estimatedgradient value, based on the initial values m₀, g_(x0), and g_(y0) ofthe estimated pixel value, the first estimated gradient value, and thesecond estimated gradient value, and the Bayes' rule-based probabilitymodel described with reference to FIG. 3.

The Bayes' rule-based probability model according to an embodiment ofthe disclosure may be obtained by modeling a probability that a pixelvalue of each pixel included in a first region around a first pixel in afirst image originates from a first pixel of an original image having anestimated pixel value, a first estimated gradient value, and a secondestimated gradient value.

The processor 1210 may optimize the estimated pixel value, the firstestimated gradient value, and the second estimated gradient value of thefirst pixel by updating the estimated pixel value, the first estimatedgradient value, and the second estimated gradient value of the firstpixel such that the probability increases.

The processor 1210 may obtain pixel values of pixels included in asecond region around the first pixel in the original image, based on theestimated pixel value and the estimated gradient values of the firstpixel.

A loss function may be determined based on a probability that a pixelvalue of each pixel included in the first region of the first imageoriginates from a pixel value of each pixel included in the secondregion of the original image. Here, the probability and the lossfunction may be represented by a function of a difference between apixel value of each pixel included in the second region of the originalimage and a pixel value of each pixel included in the first region ofthe first image.

The processor 1210 may optimize the estimated pixel value, the firstestimated gradient value, and the second estimated gradient value of thefirst pixel by updating the estimated pixel value, the first estimatedgradient value, and the second estimated gradient value of the firstpixel such that the loss function is minimized.

In addition, the processor 1210 may perform optimization on theremaining pixels included in the original image in the same manner asthat used for the first pixel, to obtain optimized values of theestimated pixel value, the first estimated gradient value, and thesecond estimated gradient value of each pixel included in the originalimage.

The processor 1210 may obtain an estimated original image based on theoptimized estimated pixel values of the respective pixels.

The processor 1210 may generate a texture map based on the estimatedpixel values of the respective pixels included in the original image andthe pixel values included in the first image. For example, the processor1210 may generate the texture map by obtaining differences between theestimated pixel values of the pixels included in the original image andthe pixel values included in the first image, respectively, andperforming filtering on the differences. Here, the texture map mayrepresent a first weight of each pixel, and the first weights for pixelsincluded in a texture region may be determined to be high, whereas thefirst weights for pixels included in a region other than the textureregion may be determined to be low.

Also, the processor 1210 may generate a curvature map based on the firstgradient value and the second gradient value of each pixel included inthe original image. For example, the processor 1210 may generate thecurvature map by computing Gaussian curvatures. An example of a methodof generating a curvature map has been described in detail withreference to FIG. 7, and thus detailed descriptions thereof will beomitted.

Here, the curvature map may represent a second weight of each pixel, anda region with high second weights (i.e., high curvatures) may have ahigh degree of or amount of curve, whereas a region with low secondweights (i.e., low curvatures) may have a low degree of or amount ofcurve.

The processor 1210 may generate a decontour map based on the texture mapand the curvature map. The decontour map may represent a third weight ofeach pixel, and the third weight may be determined based on the firstweight of each pixel represented by the texture map and the secondweight of each pixel represented by the curvature map.

The processor 1210 may generate the second image by applying, to theestimated original image, the third weight of each pixel represented bythe decontour map, applying (1−third weight) to the first image, andthen combining the first image with the estimated original image. Here,the second image may be an image from which false contours of the firstimage are removed.

The display 1230 according to an embodiment of the disclosure convertsan image signal, a data signal, an on-screen display (OSD) signal, acontrol signal, or the like, that has been processed by the processor1210, to generate a driving signal. The display 1230 may be implementedas a plasma display panel (PDP), a liquid-crystal display (LCD), anorganic light-emitting diode (OLED), a flexible display, or athree-dimensional (3D) display. Also, the display 1230 may be configuredas a touch screen to be used as both an output device and an inputdevice.

The display 1230 according to an embodiment of the disclosure maydisplay the second image from which the false contour is removed.

The block diagram of the image processing apparatus 1200 illustrated inFIG. 12 is merely an example, and embodiments are not limited thereto.Each of the components illustrated in the block diagram may beintegrated, added, or omitted according to embodiments. That is, two ormore components may be integrated into one component, or one componentmay be divided into two or more components, as necessary. Also, afunction performed by each block is for describing embodiments of thedisclosure, and its detailed operation or device does not limit thescope of the disclosure.

An operating method of an image processing apparatus according to anembodiment of the disclosure may be embodied as program instructionsexecutable by various computer devices, and recorded on acomputer-readable medium. The computer-readable medium may includeprogram instructions, data files, data structures, or the likeseparately or in combinations. The program instructions to be recordedon the medium may be specially designed and configured for thedisclosure or may be well-known to and be usable by one of ordinaryskill in the art of computer software. Examples of the computer-readablerecording medium include magnetic media such as hard disks, floppydisks, or magnetic tapes, optical media such as compact disc read-onlymemories (CD-ROMs) or digital video discs (DVDs), magneto-optical mediasuch as floptical disks, and hardware devices such as read-only memory(ROM), random-access memory (RAM), and flash memory, which are speciallyconfigured to store and execute program instructions. Examples of theprogram instructions include not only machine code, such as code made bya compiler, but also high-level language code that is executable by acomputer by using an interpreter or the like.

In addition, the image processing apparatus and the operating method ofthe image processing apparatus according to embodiments of thedisclosure may be included in a computer program product to be provided.The computer program product may be traded between a seller and apurchaser as a commodity.

The computer program product may include a software (S/W) program and acomputer-readable recording medium storing the S/W program. For example,the computer program product may include a product in the form of asoftware program electronically distributed (e.g., a downloadableapplication) through a manufacturer of an electronic device or anelectronic market (e.g., Google Play Store, App Store). For electronicdistribution, at least part of the S/W program may be stored in astorage medium or temporarily generated. In embodiments, the storagemedium may be a storage medium of a server of the manufacturer or aserver of the electronic market, or a relay server that temporarilystores the SAN program.

The computer program product may include a storage medium of a server ora storage medium of a client device, in a system including the serverand the client device. In embodiments, when there is a third device(e.g., a smart phone) communicatively connected to the server or theclient device, the computer program product may include a storage mediumof the third device. In embodiments, the computer program product mayinclude the SAN program itself, which is transmitted from the server tothe client device or the third device or transmitted from the thirddevice to the client device.

In embodiments, one of the server, the client device, and the thirddevice may execute the computer program product to perform the methodaccording to the embodiments of the disclosure. In embodiments, two ormore of the server, the client device, and the third device may executethe computer program product to execute the method according to theembodiments of the disclosure in a distributed manner.

For example, the server (e.g., a cloud server, an artificialintelligence server) may execute the computer program product stored inthe server to control the client device communicatively connected to theserver to perform the method according to the embodiments of thedisclosure.

Although embodiments have been described above in detail, the scope ofthe disclosure is not limited thereto, and various modifications andalterations by one of ordinary skill in the art using the basic conceptof the disclosure defined in the following claims also fall within thescope of the disclosure.

What is claimed is:
 1. An image processing apparatus comprising: amemory configured to store one or more instructions; and at least oneprocessor configured to execute the one or more instructions to: basedon a first image and a probability model, optimize an estimated pixelvalue and estimated gradient values of each pixel of an original imagecorresponding to the first image, obtain an estimated original imagebased on the optimized estimated pixel value of the each pixel of theoriginal image, obtain a decontour map based on the optimized estimatedpixel value and the estimated gradient values of the each pixel of theoriginal image, and generate a second image by combining the first imagewith the estimated original image based on the decontour map.
 2. Theimage processing apparatus of claim 1, wherein the first image includesa true contour which is included in the original image, and a falsecontour which is not included in the original image, and wherein thesecond image is an image from which the false contour is removed.
 3. Theimage processing apparatus of claim 1, wherein the at least oneprocessor is further configured to execute the one or more instructionsto: obtain, based on the first image, initial values of the estimatedpixel value and the estimated gradient values of the each pixel of theoriginal image, and optimize the estimated pixel value and the estimatedgradient values of the each pixel of the original image by updating theestimated pixel value and the estimated gradient values of the eachpixel of the original image based on the probability model.
 4. The imageprocessing apparatus of claim 3, wherein the probability model isobtained by modeling a probability that a pixel value of each pixelincluded in a first region around a first pixel of the first imageoriginates from a first pixel of the original image having the estimatedpixel value and the estimated gradient values, and wherein the at leastone processor is further configured to execute the one or moreinstructions to optimize the estimated pixel value and the estimatedgradient values of the first pixel by updating the estimated pixel valueand the estimated gradient values of the first pixel such that theprobability increases.
 5. The image processing apparatus of claim 4,wherein the at least one processor is further configured to execute theone or more instructions to: obtain pixel values of pixels included in asecond region around the first pixel in the original image, based on theestimated pixel value and the estimated gradient values of the firstpixel, and optimize the estimated pixel value and the estimated gradientvalues of the first pixel by updating the estimated pixel value and theestimated gradient values of the first pixel such that a loss functiondetermined based on the probability is minimized, and wherein theprobability is represented by a function of a difference between a pixelvalue of each pixel included in the second region and a pixel value ofthe each pixel included in the first region.
 6. The image processingapparatus of claim 1, wherein the at least one processor is furtherconfigured to execute the one or more instructions to: obtain a texturemap based on the first image and the estimated original image, obtain acurvature map based on the optimized estimated gradient values of theeach pixel of the original image, and obtain the decontour map based onthe texture map and the curvature map.
 7. The image processing apparatusof claim 6, wherein the at least one processor is further configured toexecute the one or more instructions to generate the texture map byobtaining a difference image between the first image and the estimatedoriginal image and performing filtering on the difference image.
 8. Theimage processing apparatus of claim 6, wherein the at least oneprocessor is further configured to execute the one or more instructionsto obtain the curvature map by computing a curvature of each pixel ofthe estimated original image based on the optimized estimated gradientvalues of the each pixel of the original image.
 9. The image processingapparatus of claim 6, wherein the texture map represents a first weightof the each pixel of the original image, wherein the curvature maprepresents a second weight of the each pixel of the original image,wherein the decontour map represents a third weight of the each pixel ofthe original image, and wherein the at least one processor is furtherconfigured to execute the one or more instructions to obtain the thirdweight based on the first weight and the second weight of the each pixelof the original image.
 10. The image processing apparatus of claim 1,wherein the at least one processor is further configured to execute theone or more instructions to: receive a third image comprising a frameimage subsequent to the first image, obtain first estimated informationincluding the estimated pixel value and the estimated gradient values ofthe each pixel of the original image, which are optimized and subsampledwith respect to the first image, obtain second estimated information bysubsampling initial values of an estimated pixel value and estimatedgradient values of each pixel of the third image, obtain differenceinformation between the first image and the third image, obtain thirdestimated information based on the first estimated information and thedifference information, obtain fourth estimated information by combiningthe second estimated information with the third estimated informationbased on a first probability that the third image originates from thesecond estimated information and a second probability that the thirdimage originates from the third estimated information, obtain fifthestimated information by performing optimization on the fourth estimatedinformation based on the subsampled third image, and generate a fourthimage from which a false contour of the third image is removed, based onan estimated pixel value and estimated gradient values included in thefifth estimated information.
 11. The image processing apparatus of claim10, wherein the at least one processor is further configured to executethe one or more instructions to: obtain a third probability that a falsecontour of the first image is included in each pixel of the first image,and obtain the fourth estimated information by combining the secondestimated information and the third estimated information based on thefirst probability, the second probability, and the third probability.12. The image processing apparatus of claim 10, wherein the at least oneprocessor is further configured to execute the one or more instructionsto: obtain an estimated original image of the third image by upscalingthe estimated pixel value included in the fifth estimated information,obtain a texture map based on the third image and the estimated originalimage of the third image, obtain a curvature map based on the estimatedgradient values included in the fifth estimated information, upscale thecurvature map, generate a decontour map of the third image based on thetexture map and the upscaled curvature map, and generate the fourthimage based on the decontour map of the third image.
 13. An operatingmethod of an image processing apparatus, the operating methodcomprising: based on a first image and a probability model, optimizingan estimated pixel value and estimated gradient values of each pixel ofan original image corresponding to the first image; obtaining anestimated original image based on the optimized estimated pixel value ofthe each pixel of the original image; obtaining a decontour map based onthe optimized estimated pixel value and the estimated gradient values ofthe each pixel of the original image; and generating a second image bycombining the first image with the estimated original image based on thedecontour map.
 14. The operating method of claim 13, wherein the firstimage includes a true contour which is included in the original image,and a false contour which is not included in the original image, andwherein the second image is an image from which the false contour isremoved.
 15. The operating method of claim 13, wherein the optimizing,based on the first image and the probability model, of the estimatedpixel value and the estimated gradient values of the each pixel of theoriginal image comprises: obtaining, based on the first image, initialvalues of the estimated pixel value and the estimated gradient values ofthe each pixel of the original image; and optimizing the estimated pixelvalue and the estimated gradient values of the each pixel of theoriginal image by updating the estimated pixel value and the estimatedgradient values of the each pixel of the original image based on theprobability model.
 16. The operating method of claim 15, wherein theprobability model is obtained by modeling a probability that a pixelvalue of each pixel included in a first region around a first pixel ofthe first image originates from a first pixel of the original imagehaving the estimated pixel value and the estimated gradient values, andwherein the optimizing of the estimated pixel value and the estimatedgradient values of the each pixel of the original image comprisesoptimizing the estimated pixel value and the estimated gradient valuesof the first pixel by updating the estimated pixel value and theestimated gradient values of the first pixel such that the probabilityincreases.
 17. The operating method of claim 16, wherein the optimizingof the estimated pixel value and the estimated gradient values of theeach pixel of the original image comprises: obtaining pixel values ofpixels included in a second region around the first pixel in theoriginal image, based on the estimated pixel value and the estimatedgradient values of the first pixel; and optimizing the estimated pixelvalue and the estimated gradient values of the first pixel by updatingthe estimated pixel value and the estimated gradient values of the firstpixel such that a loss function determined based on the probability isminimized, and wherein the probability is represented by a function of adifference between a pixel value of each pixel included in the secondregion and a pixel value of the each pixel included in the first region.18. The operating method of claim 13, wherein the obtaining of thedecontour map comprises: obtaining a texture map based on the firstimage and the estimated original image; obtaining a curvature map basedon the optimized estimated gradient values of the each pixel of theoriginal image; and obtaining the decontour map based on the texture mapand the curvature map.
 19. The operating method of claim 18, wherein theobtaining of the texture map comprises: obtaining a difference imagebetween the first image and the estimated original image; and generatingthe texture map by performing filtering on the difference image.
 20. Anon-transitory computer-readable recording medium configured to storeinstructions which, when executed by at least one processor, cause theat least one processor to: based on a first image and a probabilitymodel, optimize an estimated pixel value and estimated gradient valuesof each pixel of an original image corresponding to the first image;obtain an estimated original image based on the optimized estimatedpixel value of the each pixel of the original image; obtain a decontourmap based on the optimized estimated pixel value and the estimatedgradient values of the each pixel of the original image; and generate asecond image by combining the first image with the estimated originalimage based on the decontour map.